On Variational Bayes Algorithms for Exponential Family Mixtures
نویسندگان
چکیده
In this paper, we empirically analyze the behaviors of the Variational Bayes algorithm for the mixture model. While the Variational Bayesian learning has provided computational tractability and good generalization performance in many applications, little has been done to investigate its properties. Recently, the stochastic complexity of mixture models in the Variational Bayesian learning was clarified. By comparing the experimental results with the theoretical ones, we discuss the properties of the practical Variational Bayes algorithm.
منابع مشابه
Invariant Empirical Bayes Confidence Interval for Mean Vector of Normal Distribution and its Generalization for Exponential Family
Based on a given Bayesian model of multivariate normal with known variance matrix we will find an empirical Bayes confidence interval for the mean vector components which have normal distribution. We will find this empirical Bayes confidence interval as a conditional form on ancillary statistic. In both cases (i.e. conditional and unconditional empirical Bayes confidence interval), the empiri...
متن کاملAlgorithmic improvements for variational inference
Variational methods for approximate inference in machine learning often adapt a parametric probability distribution to optimize a given objective function. This view is especially useful when applying variational Bayes (VB) to models outside the conjugate-exponential family. For them, variational Bayesian expectation maximization (VB EM) algorithms are not easily available, and gradient-based m...
متن کاملConvergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values
We study the properties of variational Bayes approximations for exponential family models with missing values. It is shown that the iterative algorithm for obtaining the variational Bayesian estimator converges locally to the true value with probability 1 as the sample size becomes indefinitely large. Moreover, the variational posterior distribution is proved to be asymptotically normal.
متن کاملVariational Bayes In Private Settings (VIPS)
Bayesian methods are frequently used for analysing privacy-sensitive datasets, including medical records, emails, and educational data, and there is a growing need for practical Bayesian inference algorithms that protect the privacy of individuals’ data. To this end, we provide a general framework for privacy-preserving variational Bayes (VB) for a large class of probabilistic models, called th...
متن کاملGeneralized Beta Mixtures of Gaussians
In recent years, a rich variety of shrinkage priors have been proposed that have great promise in addressing massive regression problems. In general, these new priors can be expressed as scale mixtures of normals, but have more complex forms and better properties than traditional Cauchy and double exponential priors. We first propose a new class of normal scale mixtures through a novel generali...
متن کامل